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Abstract 

Sampling information using timing is a new approach in sampling theory. The question is how 
to map amplitude information into the timing domain. One such encoder, called time encoding 
machine, was introduced by Lazar and Toth in |18] for the special case of band-limited functions. 
In this paper, we extend their result to the general framework of shift-invariant subspaces. We 
prove that time encoding machines may be considered as non- uniform sampling devices, where 
time locations are unknown a priori. Using this fact, we show that perfect representation and 
reconstruction of a signal with a time encoding machine is possible whenever this device satisfies 
some density property. We prove that this method is robust under timing quantization, and 
therefore can lead to the design of simple and energy efficient sampling devices. 
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1. Introduction 

Given a function f{t) from a functional space E, sampling can be done in two different ways. 
In one approach, samples are taken at pre-defined sampling times {tn}n£Z, leading to a sequence 
{/n}nez = {/(^n)}nez- In this case, the question is when a set {tn} allows an exact representation 
of f{t), or more fundamentally, a stable representation of f{t). Examples of such sampling results 
are the Whittaker-Shannon sampling theorem [2j, where E is the set of band-limited functions 
BL{[—il.,i}]) and t„ = n^n/Q), sampling in shift-invariant subspaces [24J, or sampling of signals 
with finite rate of innovation (FRI) [25j. 

Another, dual method, is sampling based on timing. Instead of recording the value of f{t) at 
a preset time instant, one records the time at which the function takes on a preset value. Instead 
of the function itself, one may also consider the output of an operator 0[-] applied to the function. 
For example, one may record the instants where f{t) or 0[f]{t) crosses a certain threshold. 
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Examples of such sampHng methods include Logan's theorem for the representation of octave 
band functions from zero crossing [19j, various schemes generally known as delta-modulation |22j . 
as well as a method called time encoding machine by Lazar and Toth |18j that mimics the integrate- 
and-fire model of neurons. 

Why study such sampling with time schemes ? On the one hand, the duality with respect to the 
more traditional sampling at preset time makes it intriguing from a mathematical point of view. 
On the other hand, sampling by timing appears in nature, neurons being an example, and can lead 
to the implementation of very simple and low-cost sampling devices. A bucket that automatically 
empties itself once filled can be used as a time encoding machine to measure pluviometry. Finally, 
sampling by timing is potentially a more energy efficient way to acquire signals, since the basic 
elements (clocks, comparators, integrators,...) are lower power devices than high resolution analog- 
to-digital converters. 

The purpose of the present paper is to analyze one class of sampling by timing schemes, namely 
time encoding machine of which the integrate-and-fire scheme of Lazar and Toth jTH] is an example. 
The goal is to extend the validity of exact sampling by timing to broader classes of signals and 
operators, and to quantify robustness to timing quantization for these classes. By using the tools 
of frame theory and non-uniform sampling developed by Aldroubi, Feichtinger and Grochenig m [21 
[TO l 1121 [T3l I14j . it is possible to show that exact time encoding machines can be derived for a broad 
class of shift-invariant subspaces, and that these machines are robust to bounded timing noise. 

The outline of the paper is as follows. Section 2 defines general time encoding machines (TEM) 
with two exemplary cases, crossing and integrate-and-fire TEMs. Section 3 reviews some properties 
of shift-invariant subspaces (SISS). In particular, the notion of reproducing kernel Hilbert spaces 
is introduced. The main theorems linking density and invertibility of TEMs on SISS are presented 
in Section 4, as well as the fast reconstruction of the signal. Section 5 handles the special case of 
finite dimensional problems, to provide implementable algorithms. The question of stability under 
quantization noise is studied both theoretically and numerically in Section 6. Finally, some outlines 
of possible future works are given in Section 7. 



2. Time Encoding Machine 

We first extend the definition of Time Encoding Machine (or TEM) introduced by Lazar and 
Toth Hginilll]: 

Definition 1. A Time Encoding Machine is an operator T which maps a space E of real valued 
functions to strictly increasing sequences of reals: 



T: E 



{. . . < tji < tn+l < . . . 
hm t„ = ±oo 
n— >iboo 

The tn are the sampling times. We also say that T sampled at time t if t £ Tc. 

We call a Time Decoding Machine ( or TDM) an operator T> which maps an increasing sequence of 

reals into the space E. IfT>oT = Ids, T is said to he invertihle, and T> is an inverse ofT. 

The expression to sample may be confusing, for no measure of amplitude is recorded. Moreover, 
tn may not be sampled at time t„: a TEM may analyze the signal before deciding to sample at tn- 
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However, in practice, TEM are meant to encode a signal in real time, so that tn depends only on 
{{t, f{t)),t < tn}- We will only consider those types of TEMs in this paper. An important property 
of TEM is T-density: 

Definition 2. A sequence {tn}ne'Z is T-dense if 

tn+l -tn<T Vn G Z. 

A TEM is T-dense if, for every input signals in E, the output is T-dense. 

We now present two common cases of TEMs, namely crossing TEM and integrate- and- fire 
TEM. 

2.1. Crossing TEM 

The study of TEMs is mainly motivated by their design and implementation. One easy example 
relies on a test function and a comparator only. The sampling times then correspond to the times 
when the signal crosses the test function (the difference of the two crosses 0). An electronic 
implementation may be very energy efficient, for we do not need to measure amplitudes to encode 
the signal, but only the times when this amplitude is null. 

Definition 3. A crossing TEM (or C-TEM) with continuous test functions {^n}, noted CT^^^y, 

outputs the sequence {tn} = CT{$„} / such that: 

• $n n^ay be recovered from the set {ti, i < n — 1} 

• f{tn) = ^n{tn) 

• fit)^^n{t) ViG]i„_i,t„[ 

For instance, may be a rapidly increasing function, which is replaced by $n+i as soon as 
the TEM samples. In this case, the crossing TEM has a feedback, and tn-i may act as a time 
reference, so that a time shift between an encoder and a decoder will not affect the reconstruction. 
On the other hand, if ^n{t) = ^(t). Then, the output is exactly: 

CT^f = {teR, fit) = m}- 

We don't need a feedback with those C-TEMs, but the knowledge of some fixed time reference is 
important. Those notions are illustrated Figure 1. Let us introduce two implementations of such 
devices. 

We suppose that E C {f e C°(M), ||/||oo < S] with S <1. Then, we can use 

27r 

$T = COs{ — t). (1) 

According to the mean value theorem, there exists a spike in ]k{T/2), (/c + l)(T/2)[, for all k G Z, so 
that CT^j. is T-dense. Furthermore, is easily implementable with a RLC circuit, and potentially 
energy efficient. We could also have used 



^n,T = -1 + 2 , 
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(a) C-TEM without feedback 
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(b) C-TEM with feedback 



Figure 1: Diagrams representing two types of Crossing Time Encoding Machine. The device (a) 
uses a cosine as a test function, and there is no feedback. The device (b) uses a uses hnear test 
functions which are reset at each spike, it has a feedback. 



so that '^n,Titn-i) = " '^n,Titn-i+T) = 1 and ^'n,T IS continuous and strictly increasing. Again, 
the mean value theorem states that there exists t„ G + T] such that /(tn) = ^n,T(tn); 

hence CT^^^ is T-dense. This time, ^'n,T is linear, and can be approximated by the exponential 
charge of a capacitor which is reset at each spike, making it again potentially energy efficient. 

2.2. Integrate- and- Fire TEM 

The study of neurons leads to a different class of TEMs. It is shown that a neuron may be 
well- represented by an integrate-and-fire TEM [7] |llj : 

Definition 4. An Integrate- and- Fire TEM (or IF-TEM) with test functions {^n}, noted XT^^^y, 
outputs the sequence {tn} = XT{$„} / such that: 

• may be recovered from the set {ti, i < n — 1} 

• fi^)du = ^nitn) 



Therefore, an IF-TEM is nothing but a C-TEM on the integrated signal: F{t) = f{u)du. 
In addition to providing a suitable model for neurons, it may be interesting to study those TEMs 
for integrated signals since they have nicer properties, such as being continuous and differentiable. 



We will show in Section 4.3 that integrate-and-fire TEMs are adjoint to crossing TEMs. 



3. Non-Uniform Sampling and Shift-Invariant Subspaces 

Throughout this study, we will work within the context of shift-invariant subspaces. The 
purpose of this section is to review the tools related to this framework. The notion of reproducing 
kernel Hilbert space will also be recalled, as sampling theory requires the samples to be well-defined. 

3.1. Shift-Invariant Subspaces 

Definition 5. A Shift-Invariant SubSpace (or SISS) generated by the real function A(t) and of 
order p is defined by 

VP{X) := {fit) = CkX{t - k), (cfc)feez G /^(M)}. 
fcez 

We use the term shift-invariant, but only shifts over Z are considered in this definition. A special 
case is the space of band-limited functions i? := {/ G L'^{M.), Supp{f) C [— vr,7r]} = V'^isinCj^), 
where sincT^{t) = siniirt) / (irt) , and / = J^f is the classic Fourier transform: 

/oo 
fit)e-'^'dt. 
-oo 

Indeed, B is invariant under every shift. This result comes directly from the Shannon theorem |21j . 
Note that in the literature about SISS, we often find the definition: 

n 

VP{Xi, ...,Xn):= {fit) = YY1 ^^'^^^(^ - e l^'i^)} 

which extends our case. However, for the sake of simplicity, we will only consider the case n = 1. 
Furthermore, we will work mainly in the Hilbert space L^(M) (p = 2). We then have the following 
useful result: 

Lemma 3.1. Let f be in L^(C). Then, f £ V^iX) if and only if there exists a complex value 
function c{ljj) such that: 

(a) c is 2'K-periodic 

(b) c (— w) = c (w) 

(c) fiu) = c(u;) • A(a;) 
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Proof. First notice that X{t — k) = {5k * A)(t), where 6k is the Dirac in t = k. Hence, 

fit) = ck\{t -k) = E(cfc5fc) * \]{t). 

fcez fcez 

We now apply the Fourier transform for distributions : /(w) = c{(jj)\{u), where c(a;) = ^ exp(— iA:a;) 
is a 27r-periodic function. Because all the Ck are reals, we must have c(a;) = c(— a;). Reciprocally, 
if c(w) satisfies the conditions of the lemma, it's easy to check that / = F^^{c ■ A) G ^^(A) where 
J-~^ stands for the inverse Fourier transform. □ 

This theorem supposes implicitly that A is defined. We will come back to this property later. 
Because / = ^CfeA(t — k) £ ^^^(A) is entirely determined by the sequence {ck}k&z, two natural 
norms for / arise. The first one is the classic 2-norm ||/||i2 while the second one is ||/||/2 := 
||(cfc)A;ez||/2- We would like them to be equivalent, so that we may work with the countable sequence 
{ck} to represent f{t). This is given by the following property (see ^ or [20]): 

Lemma 3.2 (Daubechies [6j). The three following properties are equivalent: 

(a) The operator 

(cfc) ^ f{t) = j:ckX{t-k) 

is linear, bounded, and invertible with a bounded inverse. 

(b) The norms \\ ■ \\i^2 and \\ ■ Wp are equivalent on V'^{X), i.e. there exists < A < B such that 

V(cfc)fcez e ;2(]R), A\\{ck)\\i2 < \\J2ck\{--k)\\L2 <B\\{ck)\\p. 

(c) There exists < A < B such that A < G\{ljj) < B, where 

fcez 

Moreover, if these properties are satisfied, then V'^{\) is a closed subspace of , and the orthogonal 
projection Py2 on this subspace may be defined. 

The proof may be found in Appendix A. 
Note that the function G\ has an very important role on V'^{X), namely, if f{t) = Ylik^kX{t — k) 
and g{t) = Ylk dkX{t — k), then: 

<f\9> = ^ r c{^)<ii^) ■ Gx{u;)^du. (3) 

Jo 

This equation results directly from the proof of Lemma |3.2[ Note also that the inequality in (b) is 
common in frame theory. Indeed, it shows that {A(- — k)}k£Z is a frame of the space it spans (see 
[S] or [201 chap. 5]). 
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3.2. Reproducing Kernel Hilbert space 

In order to calculate the crossings of a function from V^^(A) with a test function, we would like 
f{t) to be well-defined for each t G M. 

Definition 6. A Hilbert space % is called a reproducing kernel Hilbert space (or RKHS) if the 
linear operators 

f ^ f{xo) 

are continuous, for all xq €z H. 

In the case of V'^{X), this condition is easily obtained whenever A satisfies some weak condition. 
For instance, if A is in the so-called amalgam Wiener space W^{C), i.e. 

{/_] sup |A(x -I- < oo, 

then Aldroubi, Feichtinger and Grochenig proved that it is the case. These authors have studied 
these spaces extensively in theory [8] [9] and in the non- uniform sampling framework [1] [2]. We 
will not review this theory for the sake of conciseness, and use the stronger condition: 



Lemma 3.3. Let A satisfies the properties of Lemma 3.2, and \{t) = 0{\t\ ") with a > 1/2, then 
V'^iX) is a RKHS. 

Proof. We write 

yt£R,\x{t)\< ^ 



{i + \t\r 

Let f{t) G V^^(A) = Ylk '^fcA(t — k). Then, for to G M, using the Cauchy Schwartz inequality, 

fito) = ( ^^M*o - fc)) ' < E • E ^'(*o -k)< ml ■ E a + \t'-k\)^- - ^ • "^"'^ 
k k k fc ^ ' ° 

for the serie + |to — is convergent for all to and a > 1/2, and the norms || • ||;2 and 

II • 11^2 are equivalent. □ 

In this case, point-wise evaluations are continuous functionals in this space. Therefore, the 
Riesz Theorem states that for all j; G M, there exists a function K^ G ^^(A) such that: 

V/ G V^{X), f{x) = <f\K.,>. 

K{y,x) = Kx{y) is called the reproducing kernel of the space. An expression of K^ is given by: 

Lemma 3.4. 

Let X{t) := J^'^i ^o^\ ) such that X{uj) - '^^^^ 



then A G y^(A) and: 

Kx{t) = ^ A(x - k)X{t -k) = YX(x- k)X{t - k). 



fcez kez 
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The proof may be found in Appendix B. 



In frame theory, {A(- — k)}!. is called the bi-orthogonal frame of the frame {A(- — k)}k- Indeed, 
we have the relations: 

< A(.-A;) I A(--/) > = 5ki. 
A related result states that we can also define: 

AiW:=-F-^(^) such that A^H = 



In this case, we can check in a similar way than the proof of Lemma 3.4 that ^ (Ai) = V {\) 
and that {Ai(t — k)} is an orthonormal basis of this space. However, because X{t) is often given 
by the impulse response of a physical device, we prefer to work with A even though Ai provides 
an orthogonal basis. If A is of finite support for instance, Ai and A may have an infinite support. 
Finally, in the case of RKHS, the orthogonal projector described in Lemma 3.2 has an explicit form 
given by the reproducing kernel: 

Lemma 3.5. Let Py2 be the orthogonal projector from L^(R) to V'^{X), then: 

V/ G {Py2f)it) = <f\Kt>= ^ < / I A(- - fc) > A(t - A;). (4) 

In particular, the k^^ coefficient of {Py2f) in the {A(- — A;)} frame is < f \ \{- — k) > . 

Proof. If / G F^(A), then {Pv^f){t) = < f \ Kt > = fit). Furthermore, if / G ^^(M), then for ah 

< f - {Py,f) \ g > = < f \ g > - Y^< f \ ~X{. - k) >< \{- - k) \ g > 

k 

= <f\g>-<f\{^<g\\{--k)>\{--k))>=Q. 

k 

Note that (/ — Py^f) is orthogonal to V'^{X), and ^^(A) is stable under {Py^). Therefore, Py2 is 
the orthogonal projector. □ 



4. Time Encoding Machine on Shift-Invariant Subspaces 

After this review of non- uniform sampling on SISS, we are ready to analyze TEM in detail and 
give a sufficient conditions for these machines to be invertible. 

^.1. General analysis 

Time encoding machines provide an elegant way to encode amplitude information into the 
timing domain. Indeed, from the output {tn}nez of a known Crossing-TEM, we can reconstruct 
the test functions ^n,T, hence recover f{tn) = ^n,T{tn)- Therefore, we can consider TEMs as a 
special method of non-uniform sampling, where the locations of the samples depend on the input 



8 



function. In order to construct an approximation of the signal, we introduce the operator V^t^y 
(noted V) defined as follows. Let 

tn—l ~l~ 

Sn - ^ 

so that Vn = [sn, Sn+i[ is the Vorono'i region of t„. Then Vf is defined by: 

oo 

Y,fitn)-lvAt) (5) 

n=oo 

where ly„ is the characteristic function of Vn- Therefore, Vf is a piecewise constant function: 
Vf{t) = f{tn) for t £ [sn, Sn+i[- The key fact is that, for a Crossing-TEM CT, we can compute Vf 
from the output {tn} = CTf- Hence, if we can invert V, we can recover the signal f{t). This is the 
case whenever — y|| < 1 on a well-defined space. ||/ — Vf\\i^2 is an integral of the difference 
between a function f(t) and its approximation by a piece- wise linear function Vf{t). According to 
the Riemann sums, we have the intuition that, provided that tn are dense enough, this difference 
should be small. The following calculations have been used by Grochenig in [12j, then by Lazar 
and Toth in [T8], or Chen, Han and Jia in [5j. We have: 



\f{t)-vf{t)\'dt = Y^ / \f{t)-f{tnrdt 



-OO 

/tn fSn + l 

m - fitn^dt + / \m - fitn)\'dt. (6) 



We then use Wirtinger's inequality: 

Lemma 4.1 (Wirtinger's inequality). Let /, /' € L'^{a, b) with f{a) = or f{b) = 0, then: 

f{ufdu < A(« - bf t f'iufdu. 

Ja 

Proof We calculate, for / G L^(0,7r/2) with /(O) = 0, 

- f{uf)du = r'\f'{u) - f{u) coi{u)fdu - r'\f\u) cot{u))'du > 0, 



and we obtain the general inequality with a change of variables. □ 
We suppose now that /' G L^, so that using Wirtinger's inequality in Q leads to 

\\f-Vf\\l,<Y,-2itn-Sn? \nt)\^dt+^{Sn+l-tnf |/'(t)Pdt. (7) 

We can see that this error depends on the differences (t„ — Sn), or (t„+i — t„). If we suppose that 
{tn+i — tn) < T, SO that {tn — Sn) < T/2 and {sn+1 — tn) < T/2, equation ([7|) becomes: 

\\f-Vf\\l, < / \f'(t)\'dt+ / \f'{t)\'dt < ^WfWh. (8) 

Finally, if / G V'^{X), and A' G L^, we can use a final inequality: 
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Lemma 4.2 (Chen, Han, Jia [5]). Let A G such that X' G and < A < Gx < B, then, 

Gx'ioj) 



yfeV\X), \\fh.<( sup ^) 



L2- 



Proof. Let /(t) = CfcA(t — A:), so that f'{t) = CfcA'(t — A;). We use the formula of the scalar 
product in ^^(A) described in O): we can write c{u)) = '^j^Ck ■ exp{—ikuj), so that: 



1 



2lT 



2 _ ^ / 1-^, ,m2 ri (, ,^2 



In a similar way, we have: 

11/11^2 = —/ \c[u)\ ■Gx\^) dw < — \ \c(iS)\ -GxK^) •( . J < I sup I 



2 

L2- 



□ 



Plugging this into ([8| leads to: 



^/Ili2<5ll/'f <5-( sup §^)'-|l/lli2. (9) 



We are now able to state our main theorem. 

Sufficient conditions for invertihility of TEMs 
We define 

£; = {A G L^, A' G l2,30 < ^ < B such that < vl < Gx{oo) < B,Gy{oj) < oo, and ^^(A) is a RKHS} 



as a shorthand to have A satisfy the conditions of Lemma 3.2 and 3.3 with the extra condition 
A' G -L^, in order to use 

Theorem 1. Let A G -E and 

■ n Gx 

r = vr • mf — , 
Gx' 

then, for all T < t, every Crossing-TEM CT which is T -dense is invertible. Moreover, if {tn} = 
CTf, an iterative reconstruction is given by the sequence: 

fi = PVfo (,0) 
fk+i = fi + {Ld - PV)fk ' ^ ^ 

where P is the orthogonal projector on V^^(A) and V = Vj^^} is defined in We have indeed: 

||/-/fc||L2<(^)'||/||L2— >0 

\ r / fc— >o 
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Proof. With those notations, using we have: 



11/ - PVf\\L2 = \\p{f - PVf)U2 = 11/ - vfU2 < ^11/11^2, V/ G V\X). 

The operator (PV) : V^{X) V^^X) verifies \\Id - PV\\ < T/t < 1 whenever T < T, hence is 
invertible. We then show that f — fk = {Id — PV)'^f by induction. We have indeed: 

f-fk+i = f-h-{Id-PV)h = {Id-PV)f-{Id-PV)fk = iId-PV){f-h) = {Id-PV)''+'f, 

so that 11/ — fkWi^ < FinaUy, because /i, hence all the fk can be calculated from the 

output {tn} of the Crossing-TEM, this TEM is invertible. □ 

In the case A = sincj^ for instance, we have A = so that Gx = 1. Then, we have 

A'(a;) = a;l[_^^^[, hence {inf Gx/Gy) = vr. Our theorem states in this case that if T < 1, then a 
T-dense TEM on band-limited functions is invertible. We find the Nyquist bound, which is (almost) 
the optimal bound we could hope for, according to the Beurling-Landau theorem. The reader may 
find a complete study of the band-limited case in [3] [15] or in the very good book [26]. In the 
general case, this bound is not optimal. Aldroubi and Feichtinger have calculated an optimal bound 
for cardinal spline spaces in [2, which is better than the one given with this theorem. Grochenig has 
also studied the case where A" G in |12j . and derived another bound using a different Wirtinger 
inequality. He also noticed that one can speed up the convergence using piecewise linear functions 
instead of piecewise constant functions for the operator V (we get a convergence in 0{f3^) instead 
of 0(a"), with /? < a). 

The operator (PV) consists in calculating a piecewise constant function approximating / G 
y^(A) on {tn}, and projecting the result back on F^(A). This theorem states that the operator 
{PV) is invertible whenever is dense enough. It links therefore density and invertibility of 
TEMs, and provides an iterative method to reconstruct the signal with an exponential speed of 
convergence. We could also have calculated the inverse directly: let / G V'^{X) be represented by 
the infinite column vector c =(..., c^, Ck+i, ■ ■ .)^ of its coefficients: f{t) = CfcA(t — k), then 

Lemma 4.3. Let M and A be infinite matrices over Z x Z with coefficients Mj^ = X{tj — k) and 
Aki = Q+^X{u-k)du. If f{t) = Y.kCkX{t-k) andg{t) = PVf = Y.k'IkX{d-k), thend = AMc. 

Proof. We first have: 

/i(i) :={Vf){t) = Y,f{tj}-'^V,{t) 

and 

g{t) = {pvfrn = {Pf^m = J2Y.f(*^y < i - ^) > - 

We check that < ly^ \ X{--k) >= J^''^'' X{u-k)du = Akj and that f{tj) = CkX{tj-k) = {Mc)j. 
Altogether, 

dk = Yl fih)- < Iv, I A(- - k) >=Y,Akj{Mc)j 
j j 
which can also be written d = AMc. □ 
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The question of whether we can recover a signal / from its samples f{tn) = < / I Kn > has 
a classic answer in frame theory: we can do so whenever there exists < A < B such that for 
all / G V^iX), A\\f\\l2 < T^fi^nf < ^||/||l2, i-e- {KtJ is a frame. In particular, if there exists 
< 5 <T such that 5 < — 1„ < then this is the case (see [1]), and we can recover the signal 
in a stable way with: 

where {i^t„} is the bi-orthogonal frame of {Kt^}. In our case, we are not using the fact that tn+i — 
tn > but the weaker form : lim„_j>-|-oo = ±oo. This may lead to an unstable reconstruction, 
but we can always deduce a stable one by erasing some extra data, according to this result. Instead 
of deleting data, we have chosen the more elegant way of weighting the samples (see [12]). We have 
actually shown that: 

A\\f\\l,<Y,Wnf{tn? = \\Vf\\l,<B\\f\\L2 With Wn = Sn+1 - Sn- 

4-3. Integrate- and- Fire TEM as the adjoint of crossing TEM 

We show in this section that IF-TEMs may be seen as the quasi-adjoint case of C-TEMs. 
Indeed, with an IF-TEM XT, we may now recover <I>n(in) = //"^^ fo{u)du from {tn}„ = ITfo- We 
introduce the operator Z = defined by 

{Zf){t):=Y, / f[u)du-Ks^^,{t). 

It is an operator from l^^(A) to L^(]R), so that PZ is an operator from V'^{X) to itself. We calculate 
the adjoint [PZ)* with the following lemma: 

Lemma 4.4. Let V' he the operator from V'^{X) to L^(M) defined by: 

then, {PZy = {PV'). 

Proof. We calculate, for /, g G V'^{\), 

< {PZ)f \g> = < P{Zf) \Pg> = <Zf \g>=Y, T"^' f{u)du < K,^^, \ g > 

= X] / f^^^ ■ ^[tnA+JW •5'(Sn+l) = / f{u){y2HnU+Ai^) g{Sn+l))du 

and we recognize the expression < / | V'g > = < / | {PV')g >, which completes the proof. □ 

The operator V' is very close to the operator V introduced previously: only the role of tn and 
Sn has been permuted. But because the only important property of V used to prove that the 
invertibility of (PV) is that Sn+i — Sn < T whenever tn+i — tn < T. Because of the tautology 
tn+i — tn < T whenever tn+i — tn < T, we can conclude as in Theorem [TJ 
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Theorem 2. Let X G E and 

r = TT • mf — , 

then, for all T < t, every IF-TEM which is T -dense is invertible. Moreover, the reconstruction is 
similar to the one of Theorem^ using the operator Z instead ofV. 

Proof. Similar to Theorem [l| Notice that {Id - PZ)* = {Id - PV), so that \\Id - PZ\\y2 = 
\\Id — PV'\\y2 < 1, and the same reconstruction method works. □ 

This duahty has been noticed by Lazar and Toth in [18] in the case of F^(sinc7r), and we have 
extended it for aU SISS using only properties from RKHS. Notice that it is also the first result 
which differs from the non uniform sampling case. Indeed, the idea to sample the integrated signal 
is natural with time encoding machines, but not with regular sampling devices. 

In the rest of the paper, we will only consider Crossing-TEM, due to this duality. 

4.4- Fast algorithm for reconstruction 

We no w pre sent an algorithm which does not need the knowledge of A to work. Indeed, according 
we have to calculate A^i = J^^'^^ X{u)du. But A may have a large support, so 



4.3 



to Lemma 

that those elements are computationally expensive to calculate. An alternative has been given by 
Grochenig in [13]. We recall that M is the matrix with elements Mj^. = A(tj — k). We suppose that 
the output {tn} is T-dense, and we note r = tt • inf(GA/GA') as in Theorem [TJ 

Lemma 4.5 (Grochenig [l3]). Let D be the diagonal matrix with elements da = Wi = Sj+i — Si, 
let M* be the adjoint of M, then U := {M*DM) is an operator from V'^{\) to itself, and for all 

T 

/iC/||y2 < max{l - /i(l - 7)2,^(1 + 7)^ - 1} with 7=-. 

r 

In particular, if T < t, with the optimal choice /x = (1 + 7^)"-*^, we find 

\\Id-^^U\\y2<p■.= -^<l. 

We included the proof in Appendix C for completeness. Note that the speed of convergence 
is slower (we have a convergence in 0(/3") instead of O(a^), and /3 > 7), but each iterative step 
is faster to calculate: the matrix A is no longer required; we only need to compute the diagonal 
matrix D and M, which is inexpensive. 

As we did before, we conclude that U is invertible, and it gives a reconstruction of the signal 
with an iterative method. We have actually proved that M is left-invertible as soon as T < r, and 
a left inverse of M may be = U~^M*D = {fiM*DM)-'^^M*D. Of course, we could have used 
the pseudo-inverse M"!" = {M*M)~^M*, but the inverse of (M*M) is more difficult to calculate, 
and less stable with respect to numerical error. We also notice that we introduced the matrix D 
to compensate an accumulation of {t„}. 
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5. Periodic case 



As we have seen, TEMs allow to encode a large class of signals in real time, without loss of 
information. However, the decoding cannot be executed in real time. We also have considered 
infinite length signals in our model, which is not practical for an algorithmic point of view. In this 
section, we focus on the finite dimensional case. For instance, the signal may be in a shift-invariant 
subspaces and i^-periodic, so that it can be described with a finite number of coefficients. The 
signal could also be with finite support, but this case can be handled as a periodic case, which is 
why we will focus on periodic signals. Studying the finite dimensional case will also allow us to 
estimate the effects of noise. 

5.1. Shift-invariant subspace and TEMs on L\ 
We will work in the Hilbert space 



■.= {f,yxeRJ{x + K) = fix), / /2(x)dx<oo} 




together with the scalar product 




The Fourier series theorem states that / G L^(M) may be written as 



fit) = J2-f^' ^^p(--^^)^ ^ith /" = f-n = < fit) I expi'^^t) > . 



We introduce, for A G L|^(M), the periodic shift-invariant subspace: 



K 



Vk{X) := {fit) = CkXit -k), Ck€R}. 



k=l 



Within this new framework. Lemma 3.1 becomes: 



Lemma 5.1. Let f G L|^(M), then f G V^(A) if and only if there exists a complex valued sequence 

{Cn}n&Z such that fn = CnXn with Cn+K = Cn and C-.n = Cn. 



Proof. Similar to Lemma 3.1 If fit) = X]fc=i CA;A(t — A;), we now have: 




K 




K 



and 



fc=i 



n=l 



□ 
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The space is now of finite dimension, hence it is always closed in L|^(M). The conditions of 
Lemma 3.2 are therefore satisfied whenever the operator 



(cfc) ^ J2k=i ^kKt - k) 

is injective. 

Because we would like to recover the signal working only on [—K/2, K/2], we suppose that 
TEMs samples only within this interval. The output is therefore {tn}n=i..j = Tf, where —K/2 < 
ti < t2 < • • • < tj < K/2. We introduce t j+i := ti + K, and we modify slightly the definition of 
T-density, to handle periodicity: 

Definition 7. An increasing sequence {ti,t2, ■ ■ ■ ,tj} is T-dense if 

tn+i-tn<T, Vn = l..J. 

With this definition, all the previous results hold: a TEM on V^{X) is invertible if it is T-dense 
on [—K/2, K/2], and we can recover the signal using iterative algorithms. We are now interested 
in a more convenient way to recover our signal. If A satisfies some additional properties, then this 
reconstruction may be done in a more efficient way. 



5. 2. A with finite support 

This method was introduced by Grochenig and Schwab in |14j . We suppose that the restriction 
of A over [—K/2, K/2] has support of size 5, and that 5 << K. Without loss of generality, we 
can suppose Supp{\) C [—S/2, S/2]. We can think of polynomial splines for instance, since they 
have a small support. Let J be the number of samples taken by our C-TEM inside [—K/2, K/2], 
and let M be the matrix of size J x K with elements Mjk = X{tj — k). Let f be the column 
vector of size J with elements f{tj), and c = (ci, • • • CnY' the vector representing our input signal: 
/W = Ef=iCfcA(t-fc). Them 

f = Mc 



and we can recover c if and only if M is left invertible. According to Lemma 4.5, this is the case 
if {tn} is dense enough, and a left inverse may be calculated by: 

= {ij,M'^DM)-^hA'Fd. 

The positive definite matrix U := DM of size K x K has elements: 

J 

Uki = ^WjX{tj-k)X{tj-l). (11) 
j=i 

Lemma 5.2. A necessary condition to have X{tj — k)X{tj — I) ^ is to have \tj — k\ mod K < S/2 
and \tj — l\ mod K < S/2. In particular, if \k — l\ mod K > S, then Uti = 0. 

Proof. This follows from the fact that the support of A is included in {[—S/2, 5/2] + KIj), and from 
the triangular inequality \k — l\ < \k — tj\ + \tj — l\. □ 
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Therefore, most of the coefficients of the matrix U are zero: U is zero except on a large diago- 
nal of size S, plus on its corners. Notice that if we work with finite support signals and not with 
periodic ones, then this property holds, without the modulo K. In this case, U has just a large non 
null diagonal. In both cases, the matrix U is invertible with a fast algorithm in 0(5'^), using the 
Choleski decomposition for sparse matrices. 



6. Robustness to noise 

With physical devices, time locations of the samples can only be recorded with finite precision, 
so that the effective recorded time t'j is different from the real time t„. This may result from 
quantization error for instance. The purpose of this section is to show that Time Encoding Machines 
are robust under this type of error. Note that we do not suppose the signal to be noisy, but only 
the locations of the samples. 

6.1. Some generalities about bounded errors 

We will study the error in the case of periodic signals. We suppose therefore that A is ii'-periodic. 
We recall the study so far in this finite case: 

Lemma 6.1. There exists r such that, for all T < t, for all t = {ti, • • • ,tj) which is T-dense, the 
matrix M{t) with elements M{t)jk = \{tj — k) of size J x K is left invertible. 

This result is independent of J. We will note in the following 

N{t) = (M(t)^M(t))-^M(t) 
to be the pseudo-inverse of M(t). In this case, the reconstruction is given by: 

c = iV(t)$(t) 

where ^>(t) = {^i{ti), ■ ■ ■ ^j{tj)) = (/(ti), • • • , /(*,/)) for a Crossing-TEM. Because of the noise, we 
would have access to t' = {t'l, ■ ■ ■ t'j), and thus to some M(t') and some $'(t'). The reconstruction 
from the noisy data is: 

c' = Ar(t')*'(t'). 

Several problems may occur. The fact that the functions may depend of the past is not 
well suited for our study, for we do not have a model to describe the effect of the noise on those 
functions. Therefore, we will study the case = $, or $'(t') = ^(f). Then, if we want M(t') to 
be left invertible, we should ensure that t' is T'-dense with T' < r. In particular, we cannot use 
an unbounded model for the noise in the timing domain. We introduce the infinite norm of t — t' 
to be: 

||t - t'lloo := sup {\tn - 4I)> 
n=l..J 

and for e > 0, we note the closed ball of center t and radius e 

B{t,e):={t' = {t[,--- ,tj), lit- t'lloo <e}. 



16 



Lemma 6.2. Let T < T' < t. If t = (ti, ■ ■ ■ tj) is T -dense, with T < t, then, for all t' £ B(t, eo) 
where eo = (T' — T)/2, t' is T' -dense. In particular, M(t') is invertible for all t' G -B(t, eo). 



Proof. We use the triangular inequality: 

t'n+l ~ < \tn+l ~ ^n+\ \ + l^n+l — ^nl + l^n " ^nl < ^0 + ^ + ^0 < 

□ 

Again, the result is independent of J. However, we may always suppose this number to be 
bounded by a constant: 

Lemma 6.3. Let t = he T-dense. Then there exists J' < 2\K/T~\ and a sequence 

(ii, • • • ij') such that t = (tj^, • • • t^/^) is still T-dense. In particular, M(t) is a suhmatrix of M{t) 
of size J' X K which is left invertible. 

Proof. With the notation tj+i = ti + K, we construct in by induction: 

k = 1 

in+i = max{j G [1 • • • J + 1], \tj - ti-^\<T } 

This sequence is constant from a certain rank J' + l, and we have tij,_^^ = ij+i = ti-^ +K. Moreover, 
with this definition, we always have tj„+2 ~ — ^- Finally, we write: 

^ = ^ijl + i ~ = i^ij' + l ~ *«(2LJ'/2J+1)) + (*«(2LJ'/2J+1) ~ *«{2LJ'/2J-1)) ^ (*«3 ~ ) > L-^ /^J " T. 

Therefore, (K/T) > [J'/2\, \K/T'] > J'/2 and J' < 2\K/T']. □ 

Because M(t) is left invertible, of size J' x K, we can use it to reconstruct the signal: c = 
A^(t)<I>(t). Thus, we can always suppose the number of lines of M to be less than 2\K/2'] without 
loss of generality. Note that because M is full column rank, there exists K lines such that the 
K X K resulting sub-matrix is invertible. We could have concluded that this sub-matrix is still 
invertible whenever t is taken in a small neighborhood, for the set of invertible matrices is an open 
set. However, we do not have any expression of this neighborhood. This is why we have chosen 
to take more lines, to guarantee T-density, and therefore left-invertibility on a large neighborhood, 



according to Lemma 6.2 



6.2. Robustness under bounded noise in the timing domain 

In sampling theory for band- limited signals, the error introduced by quantization in the am- 
plitude domain leads to an error MSE[X' , X) = 0{e'^), where X' is the reconstruction from the 
noisy data, X is the original data, and MSE stands for the Mean Square Error: MSE{X' ,X) = 
\\X' -X\\j,. Thao and Vetterh have shown in |23] that we also have MSE{X',X) = 0(r-2) where 
r denote the oversampling rate. This theorem has been extended by Chen, Han and Jia in [5] 
for non-uniform sampling in shift-invariant subspaces. Using the mean value theorem, these two 
articles use the fact that having oversampling is equivalent to having infinite precision of the am- 
plitude of some f{tn), where the locations {tn} are not known precisely (but are more and more 
precise with the oversampling factor). Here, we are working in the framework of TEMs in a finite 
dimensional space. In this case, we also have MSE{X',X) = 0{e^). The calculations are similar 
to the ones in |23j . 
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Theorem 3. Let A be of class and K -periodic. LetCT^ be a C-TEM on V^(A) without feedback 
which is T -dense, with T < t = ^'^{{Gx/Gxi). Let T' such that T < T' < r and eo = (T' — T)/2. 
Then, there exists a constant a which depends only on X,^,K and T' such that, for all output 
t = {ti, ■ • - tj) of CT^, for all t' = {t[, ■ ■ • t'j) £ B{t, eo), we have: 

||iV(t')^(t') - iV(t)$(t)||,2 < a||t' - t||oo. 

Proof. According to Lemma [6.3[ we can always suppose J = Jq = 2\K/T]. We introduce e = 



||t' — t||oo < eo- Let Ss ■= {t = (ti, • • ■tjg),t is increasing, t is (5-dense}. According to Lemma 6.2 

ST + B{0,eo) C St' 

and St' is compact (it is closed and bounded in M"'"). Lemma 6.2 ensures that A^(t) is well-defined 
in St + -8(0, eo), and its expression is: 

iV(t) = {M{tfM{t))-^M{tf. 

In particular, the coefficients of A^(t) are continuous with respect to the coefficients of M(t). 
Therefore, if A is of class C^, so are the coefficients of -A^(t). Actually, if we denote by Ni^j{t) 
the coefficients of iV(t), and Flj{t) = J-A^fci(t) the coefficients of F*(t) = J-A^(t), then all those 
functions are continuous on the compact St'- Thus, we can find a constant ai such that: 

Vt eST', I Nkj (t) I < ai and | F^- (t) | < ai . 

For a matrix A = [Aij) of size K x Jq, and for b G M.'^'\ using the Cauchy-Schwartz inequality, we 
have: 

P^ll'^ = E(E^^.^.)' < E(i;4) • (E^') <K-j'o- (max4) . (max62). (12) 

1=1 j=l i=l j=l j=l 

We also have the Taylor inequality: 

mfi) - $(ti)| < \t'i - ti\ ■ sup $'(t) < e||^>'||oo (13) 

and 

Jo 

I Nik (f) - Nij {t)\<Y,\ t'j sup I F4 (i) I < ai Joe. (14) 

j=i te{t,,tr) 

We now write 

\\N{t'Mt') - Nmmp < \mt')mt') - m) y + ii {mt') - N{t))my- 

Using inequalities (12) and ( |13[ ), the first term is bounded above by: 

\\N{t'){^t') - Ht))\\l < •maxiVi,(t)2 .max($(t'J - ^ti)f < {KJ^alW^'WDe^ 

The second term is bounded in the same way with (12) and (14): 

\\{N{t') - N{t))<^>{t)\\l < KJl ■ max(A^i,(t') - N^j{t)f ■ max<^>{tif < (KJoXll^llL)e'- 
Altogether, we proved that 



||iV(t')$(t') - A^(t)<I>(t)||,2 < a • e with a := VKJoai^WWlo + JolMl 

18 



□ 



6.3. Numerical Results 

We have implemented the method described above with the fohowing parameters. We took 
f{t) = X^^x CkX{t — k) with K = 50, A the cardinal spline of order 3, and Ck are chosen uniformly 
at random in [—1, 1]. Then, we normalized f{t) so that ||/||oo = 1- We encode such a signal with 
the Crossing-TEM CT^ with: 

^>(t) = Q • cos^27rt^, 

where a = 1.1. It is easy to check that this TEM is 1-dense, and therefore is invertible according to 
|3]. The crossing times (ti, • • • , tj) are then recorded with a quantization error e. From those points, 
we calculate the matrix M with elements Mj^ = A(tj — k), and the vector f = (<I>(ti), • • • , 
and we recover c = {ci, ■ ■ ■ ck) with c = M^f. We then have the reconstructed signal f{t) = 
^k=i ^k^it ~ k). We then recorded the error between / and / for various /. In Figure 2, we 
plotted the mean and the 95% confidence interval of the error for 1000 different signals f{t), 
with respect to the P norm of {in — in}n=i..Jo- We recall that because the number of samples is 
finite, this P error is equivalent to the error. The choice of the P quantization error leads to a 
linear behavior. Therefore, the error of the signals with respect to the quantization error of 
the time locations is sub- linear, as Theorem [3] stated. 




Figure 2: The error of the signal versus the P quantization error of the time locations. 
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7. Future Work 



The tools we used to prove the invertibihty of dense C-TEMs are the following facts: ^^(A) is 
a RKHS, there exists an orthogonal projection on this space, we can use Wirtinger's inequality on 
this space, and finally, there is an inequality of the form: 

V/GF2(A),||/'|U2<C||/||i2 

saying that the map: 

D : V^{X) CL^ L2 
/ ^ /' 

is continuous. Therefore, similar results may be derived on a space E where those requirements 
are satisfied. In particular, if we introduce the standard Sobolev space in dimension one: 

= {f e f G L""} 

with its norm = + ||/'||2^ = ^||(1 + oj^) f {u;)\\l„ we have the: 

Theorem 4. If E is a closed subspace of such that the norms and \\.\\i,2 are equivalent 

on E, i. e. there exists a constant C such that: 

yf(^E, \\f'\\L2<c-\\f\\L2, 

then, every C-TEM (and IF-TEM) which is T-dense, with T < t := tt/C, is invertible. 

Proof. Because E C C is closed, we already have the existence of an orthogonal projector 
on E. We can use the Wirtinger's inequality for E G L^. Finally, ii^ is a RKHS because is a 
RKHS (with kernel Kt{u) = e-l*-"l/2)- This comes from: 

l/WP = ^-| / f{u^)e'^'dt\<^ I Y^du- /(l+^')l/HPd^<vr||/||2,,. 

Finally, the exact same proof of Theorem [T] and Theorem [2] can be derived, which gives the result. 

□ 

Therefore, our result about TEMs on SISS can probably be extended in this direction. However, 
while it is easy to show that V'^{X) satisfies the conditions of Theorem |4| it may be more difficult 
to extend it to some other cases. Moreover, the case of SISS allowed us to deduce the algorithms 
for this case. Notice also that the condition of Lemma 3.3 to ensure that V^^(A) is a RKHS is not 
necessary, for it is always satisfied whenever: 

30 < A< B, A <Gx{uj) < B, and Gx'{u}) < C < oo. 

Another direction to look at for future work is the behavior of the error. While we showed 
that the error is sub-linear in the quantization error, we noticed that this error appears to be 
actually linear in the P quantization error. We could not prove this fact, nor could we give an 
explicit bound of the decay of the error. 

A final direction, and probably the most difficult one is the following remark. If A has finite 
support [0,5], then all the elements of the matrix M are zero, except on a large diagonal, and 
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the matrix M is left-invertible. Suppose that the TEM has sampled J times during the interval 
[ki, ^2]) then the value of the function on this interval depends on K = k2 — ki + S coefficients. 
In particular, if the TEM has density T < 1, and if the interval is large enough, we can ensure 
that J > K, so that there are more equations than coefficients involved. But we cannot conclude 
that the sub-matrix involving those equations and coefficients is still left-invertible. Aldroubi and 
Grochenig showed that it is the case for cardinal splines SISS, because of the very special structure 
of those functions This would allow us to reconstruct the signal independently of the past, and 
therefore provides us better algorithms for reconstruction, or bounds for the error. 



Appendix A. 

Proof of Lemma \3.S\ (a) (b): Let ^ be the map described in (a). The right-side of (b) indicates 
that ^ is bounded, and the left-side that its inverse is also bounded. Then, because V^i^X) is the 
image of the closed subspace /^(M) by the linear continuous function it is closed, and because 

is an Hilbert space, the orthogonal projection may be defined, 
(b) ^ (c): We introduce c(a;) as in Lemma 3.1 Using Parseval's equality and the 27r-periodicity 
of c, we have: 



1 -1 pco -1 pliv 

h = ^\\f\\l- = ^ \c{u).X{co)\'du = - \c{u:)\'-Y,\X{c. + 2kn)\'du 

J~oo Jo 



27r 

\c{u)\'^Gx{ojfdu} 







1 

Now, using again Parseval's equality for Fourier series, 

r2TT 



k&z •'^ 



If A < G\{uj) < B for all uj G [0, 27r], then the inequality in (b) holds. Reciprocally, if the inequality 
is not satisfied in some neighborhood of wq, then we may take c to satisfy the properties of Lemma 



3.1 , with support in this neighborhood, and strictly positive. This particular choice contradicts (b). 



□ 



Appendix B. 



3.1 



we only need to 



Proof of Lemma 3.4- We first check that A G V'^{X). According to Lemma 

check that G\ is 27r-periodic, and that G\{—ijj) = G\{u}). The first part comes directly from the 
definition: 

Gl{uj) = Y,\\{u: + 2k7:)\\ 

fcez 

and the second part from the fact that A is a real valued function, and therefore satisfies A(— cj) = 
A(a;). We now remark that, for all / = Ylk '^k^i't — k) £ y^(A), equation (3) reads: 



</l A>= 



1 

2^ 



2lT 



Gl{uj)du 
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1 

2^ 



27r 



c{uj)dijj = Cq. 



We deduce from this: < / | A(- — A;) > = Ck, and we can write 

f{x) = Y,CkK^ -k) = Y,<f\K--k)>\{x-k) = < / I A(- - k)\{x -k)>. 



This gives us the first equaUty 



Kiy, x) = K,{y) = ^ A(y - k)X{x - k). 



Now, let ci{uj) satisfies the conditions of Lemma 3.1 Then, £2(0;) := ci(a;)G^(a;) also satisfies the 



conditions, and 

C2(ti;)A(a;) = ci(a;)A(a;). 

Therefore, Lemma 3.1 states that V'^{X) = V^{X). Applying the same calculations in V^(A) leads 



to the second equality 

Kiy, x) = K.,{y) = A(y - A;)A(x - k). 



Appendix C. 

Proof of Lemma Let us compute M*. For d = in we have: 



□ 



<Mc\d>= Y,fiin)dn = Y,<f\Kt„>dn=<f\ Y^d^Kt^ > 



n n 



SO that M*d = '^ndnKt^. Therefore, Uf = ^nWnf{tn)Kt^. Because the operator U is self- 
adjoint, so is [Id — fjiU). We want to calculate the maximum (resp. minimum) eigenvalue of 
{Id — fJ-U). This is given by: 

lmax-= sup < (Id- fiU)f \ f > = sup \\ff - fJ^ < U f \ f > . 

We have < C// | / > = E„ "'n " < KtJ f > = En^nfitn? = WVfWl^. With the triangular 

inequality and with 



\l2 -\\Vf\\L2 <\\f -Vf\\L2 <^\\f\\L^. 
Altogether, we find: 

Ima. < sup ll/f - ^(1 - 7)'||/f = 1 - ^(1 - 7)'- 

/GVM|/||^2=1 

With similar calculations, we find Imin > 1 — /^(l + 7)^- Hence, \\Id — fJ-U\\y2 < max{l — fi{l — 
7)2,Ml + 7)'-l}- 

These two quantities are linear in /x. The optimal value with respect to /U is obtained when those 
quantities are equals. This leads to /i = (1 + 7^)~^, and in this case, 



1-Mi-^) =Mi+7r-i 



1 + 7 

Finally, this expression is less than 1 if and only if 7 < 1 . □ 
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